Parameter sharing

Parameter sharing is a nice strategy to reduce the model size by incorporating the natural structure of the data (often into the Neural network models) in Machine learning.

For instance, Convolutional neural network‘s convolutional layers learn a small number of parameters that express a “filter” that is shared across the visual field.

In LLMs, ALBERT is an example.